profiles: split out SDK and JFR shim modules#8207
profiles: split out SDK and JFR shim modules#8207jack-berg merged 7 commits intoopen-telemetry:mainfrom
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8207 +/- ##
============================================
+ Coverage 90.27% 90.31% +0.04%
- Complexity 7684 7725 +41
============================================
Files 850 850
Lines 23186 23259 +73
Branches 2352 2364 +12
============================================
+ Hits 20931 21007 +76
+ Misses 1530 1528 -2
+ Partials 725 724 -1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| * | ||
| * <p>This class is not threadsafe and must be externally synchronized. | ||
| */ | ||
| public class JfrLocationDataCompositor { |
There was a problem hiding this comment.
The exporter module should only have exporter functions in it. Eventually, when OTLP profiles stabilize, this will be merged into opentelemetry-exporter-otlp, at which point it won't be acceptable to have JFR artifacts.
This JFR stuff seems best described as a shim or bridge, like https://github.com/open-telemetry/opentelemetry-java/tree/main/opencensus-shim or https://github.com/open-telemetry/opentelemetry-java/tree/main/opentracing-shim.
The question is whether it made sense for it to be hosted in this repo or elsewhere - i.e. opentelemetry-java-instrumentation or opentelemetry-java-contrib. And whether there is anything in common across profile instrumentations (i.e. shims / bridges) that should be extracted to a common place, like a profile API / SDK module.
I haven't seen ProfilesDictionaryCompositor before just now. It looks like it was added while I was OOO. This seems to be the "anything in common across profile instrumentations" bit, and is what allows this JFR piece to be so small / compact.
There was a problem hiding this comment.
Thanks. I think you're right conceptually, but the situation is muddied a little by a couple of factors.
JFR is part of the JDK. Unlike the other bridges, it's not brining in additional dependencies. Also, profiling doesn't have a separate API module, so moving the JFR pieces elsewhere still leaves them close coupled to the not-really-public interface of the exporter.
As you noticed, as much as possible of the code doesn't use JFR APIs, so the JFR layer is quite thin. However, it's using naming and design patterns that are the same as in the exporter utility code and for easy of use should be kept in sync with them. If the JFR layer moves elsewhere, I'd be tempted to pull out the bits that are not JFR specific and yet also not part of the normal exporter pattern used by other signal's exporters and move those too. That's probably the DictionaryCompositor and SampleCompostion bits in addition to the JFR package.
I think my preferred option would be to keep the JFR and non-JFR not-exporter pieces a) together and b) in this repo, but in a separate module from the exporter. Splitting them up makes it harder to keep design consistency between them and moving them to a different repo makes it harder to keep them in sync with the exporter. However, both those problems should lessen over time as things stabilise, so maybe the short (relatively!) term pain is worth it to get a better long term structure, which would point more towards moving bits into e.g. contrib.
There was a problem hiding this comment.
JFR is part of the JDK. Unlike the other bridges, it's not brining in additional dependencies. Also, profiling doesn't have a separate API module,
Yeah we don't have any precedent for this type of thing to lean on. To reiterate one thing I've said previously, logs started out with a design similar to this, with a goal of only having an SDK for bridging support where the bridges would directly depend on the SDK. It turned out to be non practical and so we separated out a thin log API.
One practical thing we could do:
opentelemetry-sdk-profiles: Home ofProfileExporterand ease of use utilities likeProfilesDictionaryCompositor. Its reasonable to want to have alternative profile exporters besides OTLP. Minimally things like an OTLP logging exporter, but also maybe someone would want to build a pprof exporter. Having a dedicated profile SDK module allows you us to build such things without them needing to depend on OTLP.opentelemetry-jfr-profile-shim: Home of the JFR translation bits. Since there's not a lot to this, I wouldn't be heartbroken if this was merged intoopentelemetry-sdk-profiles, but I do think that some logical separate between shared bits and implementation specific bits is useful. I agree with you that its useful to keep the JFR bits close to theProfilesDictionaryCompositorso they can be developed in tandem, but someday after things stabilize, it may make more sense to move JFR out to a dedicated module inopentelemetry-java-instrumentation. In this case, it will benefit from having been split out.
Thoughts @open-telemetry/java-approvers?
There was a problem hiding this comment.
@jack-berg ok, take a look at this and see if it's along the lines you intended. If so I'll continue and factor out the JFR parts too. That will have to happen - they need the exporter, but that would create a circular dependency if they stay where they are.
There was a problem hiding this comment.
I like it! Left a few comments.
8b9b62a to
8626bff
Compare
| * of a JFR recording file. This is not a supported CLI and is not intended to be configurable by | ||
| * e.g. command line flags. | ||
| */ | ||
| public class JfrExportTool { |
There was a problem hiding this comment.
Since this is just a demo, should move somewhere else, like the test module. Not important at the moment since this isn't published yet. A TODO in the build.gradle.kts file above the commented // id("otel.publish-conventions") line should suffice for now.
| // and therefore a bit of a pain to get gradle to compile against... | ||
| compileJava { | ||
| sourceCompatibility = "1.8" | ||
| targetCompatibility = "1.8" |
There was a problem hiding this comment.
TODO: I need to take a closer look at this. Why does it work? Is there a more idiomatic way to achieve?
There was a problem hiding this comment.
I think that was from Lauri in #7741 (comment)
When the JFR bits get moved into their own module it will go with them, or we can just drop it and declare the JFR functionality as available only on 9+ which probably won't upset too many people.
jack-berg
left a comment
There was a problem hiding this comment.
Couple of small nits, but looks good.
| * | ||
| * @param profilesDictionaryCompositor the underlying storage. | ||
| */ | ||
| public JfrLocationDataCompositor(ProfilesDictionaryCompositor profilesDictionaryCompositor) { |
There was a problem hiding this comment.
Does it make sense for JfrLocationDataCompositor to create its own instance of ProfilesDictionaryCompositor and expose a getter for ProfilesDictionaryData?
There was a problem hiding this comment.
There are use cases where external things need direct access to the mutable state, not just the result. It could still create the compositor, but would have to expose a getter for it. I originally modelled this as 'JfrLocationDataCompositor extends ProfilesDictionaryCompositor' which is in some ways cleaner, but I'm not wild about ProfilesDictionaryCompositor being a cross-module extension point in that way.
There was a problem hiding this comment.
Ok sounds good. Can learn as we go and refine if needed.
but I'm not wild about ProfilesDictionaryCompositor being a cross-module extension point in that way.
Yeah I prefer delegation to extension in most cases
Sample code for converting and exporting JFR ExecutionSample events using OTLP profiles signal.